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Abstract 

Reflective practice has been adopted by many teachers, including dissertation advisors 
who provide 1:1 teaching of research students. This paper discusses issues arising from 
our use of qualitative data analysis software (QDAS) for reflective practice and provides 
an analysis of the thematic coding and word/pattern searches of feedback given to 
postgraduate research students. We subjected written feedback provided by one 
dissertation advisor to three different analyses: thematic, text-string and text-pattern 
search. We found that use of QDAS in a thematic analysis of feedback provides new 
insights by indicating areas of strength and areas for improvement. Text and pattern 
searches are more useful in flagging students with specific learning issues. 

Key words: Reflective practice, Postgraduate supervision, Reflexivity, Teaching 
research, Research advisor 


Introduction 

Postgraduate training programmes involve teachers who engage in one-on-one teaching 
in graduate programs that lead to degrees such as PhD, Master's or in some countries, 
Honors level. In Australia, the term 'advisor' or'supervisor' is used to denote the teacher, 
whereas in other countries, the term "dissertation advisor" is used; and refers primarily 
to the chair of the dissertation committee (Spillett and Moisiewicz, 2004). Ideally, the 
teacher-student relationship is collaborative, developing from negotiated, mutually 
satisfactory arrangements between the student and the advisor (Zuber-Skerritt & Roche, 
2004). 

A major way in which the teacher can accommodates individual student needs is through 
reflective practice. The teacher uses reflection as a tool to improve his/her practice 
through a process of observation and thought (Schon, 1987). Thus reflectivity entails 
the teacher continuously questioning the validity and efficacy of his/her actions (Ostorga, 
2006), with a view to improving teaching method (Bintz & Dillard, 2007). 

This research arose during discussions with colleagues who were concerned about how 
they could demonstrate intangible skills such as reflectivity of practice. There is ample 
evidence that reflective practice leads to improved teaching of postgraduate research 
(Bleakley, 1999). The problem is that, even for the most reflective practitioners, it is 
difficult to provide evidence of reflectivity. Indeed, unless the actual practice of 
reflectivity is made explicit, there is a danger that it will remain little more than a device 
for according the appearance of academic rigour (Allen, 2004). There is very little 
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written in the literature about how to conduct and demonstrate reflective practice 
in postgraduate supervision. 

We felt that it would be possible to apply a systematic approach, such as is used in 
qualitative research, and use qualitative data analysis software (QDAS) to identify the 
key components of research supervision of one of the authors. The aim was to a) clarify 
the substantive feedback a dissertation advisor provided to students, b) identify areas in 
which this individual's practice could be improved, and c) determine if QDAS provided an 
effective means of demonstrating reflective practice. 

This paper discusses issues arising from the use of qualitative software (NVivo) for 
reflective practice and provides the results of the thematic coding and searches of a 
dissertation advisor's practices. We had available to us for analysis, hundreds of pages 
of written feedback which had been provided to students over a number of years. 

This paper indicates the extent to which the tools of qualitative research can be applied 
to facilitate reflective practice. 


Review of the Literature 

Although there are many variables that affect completion of a postgraduate thesis, the 
individual dissertation advisor is critical in all aspects of the supervision and research 
training experience (Marsh, Rowe, & Martin, 2002). The literature has identified many of 
the personal and professional attributes of effective supervisors, including "advisor style" 
which describes the general approach of advisors to both the task of research supervision 
and to the relationship between teacher and student (Hammick & Acker, 1998). Zuber- 
Skerritt and Roche (2004) detail indicators of effective teacher style as the advisor 
knowing what he/she wants, being encouraging, insightful, resourceful, committed to the 
student and directed by the student's needs. Other attributes associated with effective 
research supervision include ensuring that assessment and examination processes 
maintain high academic standards (Cryer & Mertens, 2003; TWIG Writing Group, 1996) 
and being aware of the dissertation process at the institution. This includes having an 
understanding of all the schedules, deadlines and personnel responsible for each part of 
the dissertation process (Lenz, 1997). 

The effect of these multiple demands is that research advisors can find themselves in the 
difficult position of having to provide encouragement and support to a student while 
enforcing institution rules and deadlines. Spillett and Moisiewicz (2004) portray the 
potentially contradictory roles of student supporter and institution policeman as 
comprising four separate roles. These include the Cheerleader, who demonstrates belief 
in students' abilities and encourages the student's effort, and the Coach, who builds 
research skills. The other roles are the Counselor who removes blocks, and the Critic, 
whose job is to build the student's sense of ownership, the goal of which should be 
professional socialization in the roles they will need for employment and career 
(Benaquisto, 2000). However, little guidance is offered to prepare supervisors for these 
multiple roles, and it is often left up to the individual to develop an effective teaching 
style through trial, error and self-reflection (Spillett & Moisiewicz, 2004). Indeed, it is 
through self reflection that Zuber-Skerritt and Roche (2004) and Benaquisto (2000) 
arrived at their conceptualisations. 

Most teachers and supervisors probably would say they are reflective, and that this 
improves their practice. However, would a methodologically structured self-analysis yield 
benefits that were not available through less formalized means? Could QDAS facilitate 
this, and would it be worth the effort? 
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QDAS is designed to enable analysis of language at the word, sentence or 
paragraph level. It also can be used to search for particular words or phrases. The 
underlying logic of coding and searching for thematic material is very similar to manual 
techniques (Thompson, 2002). Text-string searches match exact words or phrases and 
text-pattern searching enables searches for words with similar meanings (Durian, 2002). 

There are several advantages to QDAS. These include an ability to deal with large 
amounts of data, and a reduction in the amount of time needed for manual handling 
tasks (Thompson, 2002). QDAS also provide a more visible audit trail in data analysis 
because segments of data are not likely to be lost or overlooked during retrieval (St John 
& Johnson, 2000). 

However, there are also disadvantages to the use of QDAS. These include the financial 
cost of obtaining a software licence and the need for training and practice in order for 
users to become proficient of the program. Another limitation is described by Thompson 
(2002) who feels that there is a danger that the program can drive the analysis rather 
than vice versa, which can lead to cutting corners and findings that lack credibility. 
Finally, Welsh (2002) argues that many social science researchers do not have the 
expertise to make informed assessments of the different software choices, thus decisions 
about the program may be based on things other than the program being the best one 
for the job. 


Purpose 

The aim of this study was to evaluate QDAS, specifically NVivo, in facilitating and 
formalising reflective practice. We used a case-study self-report approach which has 
precedence in the literature (see for example Benaquisto, 2000; Spillett & Moisiewicz, 
2004; Zuber-Skerritt & Roche, 2004), in which we approached a dissertation advisor's 
documented feedback to students as data. This paper is the result. 


Method 

This study utilised a qualitative paradigm in which text rather than numbers is used as 
the unit of analysis. In this study, the students to whom the feedback was provided 
were research higher degree students. In Australia, postgraduate students complete an 
extensive thesis at either the PhD, Master's or Honors level (Marsh, Rowe, & Martin, 
2002). For the PhD and Master's, there is little or no formal course work and the focus is 
a final thesis (dissertation) that is externally examined. Postgraduate honors students 
are enrolled in an extra year, dedicated to a research thesis, at the end of their three 
year bachelors degree. For this study, we utilized written feedback provided to eight 
former research students from honours through to PhD levels. The theses were on a 
variety of topics using qualitative and quantitative methods. The feedback was 
predominantly via email, which the supervisor prefers because it is efficient, quick and 
provides a complete record of interactions. We also included written notes provided on 
hardcopies which students who preferred face-to-face meeting returned to us for the 
purpose of this analysis. 

A thematic analysis is conducted in three broad stages and is based on the assumption 
that people share some common experiences during specific events, such as being a 
postgraduate research student. In order to identify these common experiences via a 
thematic analysis, firstly, significant words and phrases are highlighted on the transcripts 
(Van Manen, 1997). The second step of analysis consists of labelling and categorizing 
these highlighted "codes". Codes that pertain to similar categories are grouped together. 
Thus, in Figure 1, the sub-themes of'encouragement', 'reframing', 'avoiding pitfalls' and 


https://doi.org/10.2042 9/ij sotl.2008.020208 


3 



Computer Facilitated Reflective Practice 


'scaffolding' all contribute to the way in which the advisor engineers success for the 
student. Because these sub-themes were developed from direct quotes ("codes") from 
all transcripts, we can conclude that the theme of 'engineering success' is common for all 
students of this particular advisor. The provision of the quotes/codes in this paper 
provides a means of validating/clarifying the analysis. 

The final stage of the analysis is to categorize the sub-themes into themes. In this case, 
three themes of'product', 'process' and 'investment' reflected how this dissertation 
advisor conceptualized her teaching. This type of analysis is an iterative approach 
(Colaizzi, 1978; Grbich, 1998; Moustakas, 1990; Patton, 2002; Van Manen, 1997) in 
which already analysed transcripts are revisited every time a new code is identified in a 
subsquent transcript. 

Enumerating the Codes 

In addition to the content analysis two other similarly focused, but more quantitative, 
analyses but were conducted consisted of a text-string (word) and a text-pattern 
(phrase) search of key words and phrases that emerged though the first analysis. For all 
analyses, the QDAS NVivo (QSR International Pty. Ltd., 1999-2000) was used. 

Raw Data 

In order to develop a dataset for analysis, we copied the comments into new documents, 
one new document for each student. We ended up with eight documents, each one 
including all written feedback to an individual student, in temporal sequence from the 
beginning to the end of the supervision process. We then treated these documents as 
transcripts and subjected them to thematic analysis. 

Rigor (reliability and validity) 

Credibility of the analysis was established through having two of the supervisor's 
students read the analysis and comment on the extent to which it reflected their 
experiences. Re-analysis of the transcripts based on their feedback led to revisions of 
the analysis. For example, the theme, "telling it like it is" was originally coded as "ticking 
off the student", which one student felt conveyed a harshness that was not part of their 
experience with this advisor. In addition, the final analysis was reviewed by a university 
professor with over 40 years teaching experience. Fie concurred with the themes and the 
analysis. Finally, the second author of this paper examined the transcripts and emerging 
themes and collaborated in identifying quotes that exemplified the themes described in 
this paper. 

Ethics 

Due to reasons of confidentiality, we did not analyse the feedback given to students from 
the advisor's current place of employment. In addition, details that might identify a 
specific research topic or institution have been changed to more generic terms. 

Results 

From the thematic analysis, 38 categories and sub-categories of feedback were 
identified. These were grouped into three main themes; the process, the product and 
the investment. 

The Process 

The majority of postgraduate supervision activities related to the process of teaching 
research. This involved three major activities of groundwork, systematization of 
workload and building success into the process. Figure 1 provides concepts that 
contribute to this theme. 

Groundwork was very noticeable early in the research supervision process. It included 
activities such as "talking the same language". This includes instructions on how to use 
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the 'track changes' function of Word, and how the student could get the best use 
out of the supervisor's time; "when doing so much cutting and pasting of sections, it 
makes it hard (for me at least) to follow." 

We found that it was very important for the advisor to demonstrate respect for the 
student as part of the groundwork. It was very clear that having passed a research 
methods class did not provide students with all the skills they needed, and this left them 
feeling inadequate. Demonstrating respect for the students would let them know that 
the supervisor would not belittle them and would take their questions seriously. This 
enabled the individuals to settle quickly into the give and take of teaching and learning 
research, which included an acknowledgement that sometimes they know best: 

"You're right...I was thinking about an alternative way to cut the budget down 
when I emailed you and I got confused. I'm not mad at you and I'm sorry if I 
sounded that way. 

Finally, setting the groundwork included communicating pragmatic issues involved in the 
administrative requirements of the university such as where to find the plethora of forms, 
policies, procedures and deadlines that apply, and the timing of the sampling frame: 

Now would be a good time to use the electoral roll as we have just had an election 
and they do lots of advertising to make sure people are registered at their current 
addresses. 

A surprising amount of feedback involved 'systemization', or getting systems in place, 
which cut down time-wasting activities. This included preparing the student for the next 
step of the research before they got there, 

Well, the [examiner'sy report [of your proposal] was pretty much as expected, so 
we just have to wait for the other report. I think it is important to know that you 
DO NOT have to re-write your proposal. The suggestions are more about what 
you need to do for your thesis. Therefore, while you wait for the second reviewer, 
think about starting your chapter 1 of your thesis. 

Systemization also entailed ensuring that the student was clear on conceptual issues so 
that literature searches, for example, were focused; solving problems promptly and even 
reprimanding students by holding them account for substandard efforts such as sending 
me rambling drafts that they hadn't bothered to edit. 
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Figure 1 . Elements of The process 





Conceptual issues 

Another aspect of systemization occurred in the early clarification of conceptual issues. 
The supervisor often used metaphors to explain these to students: 

Your intuitions and own experiences are an integral part of qualitative research 
and your use of language has to reflect that your decisions about methodology 
came about because of your deliberations ... I think an analogy is like a sculptor 
who is technically excellent but is able to breathe life and feeling into the lump of 
clay he has been given. Feeling without technique leads to a bad sculpture. 
Technique without insight leads to bland perfection. Technique and insight lead to 
art. 

We found that 'solving problems' was part of systemization because the supervisor 
expected problems in student research and had strategies to nip these in the bud. 

Finally, part of the system was to provide feedback that kept the student on track. 
Occasionally this required what we have called "telling it like it is". 

A full time master's student should spend 37 hours per week on the thesis. A part 
time student is expected to spend 18 hours a week. At this stage, I'm not sure 
whether the poor quality of your work is due to the fact that you are not spending 
the amount of time on your thesis that you need to, or that you really don't 
understand how to write a literature review. If it is the former, I suggest you 
spend the amount of time required, or withdraw... 


and 


I have spent 3 hours trying to figure out what you are saying. It was so 
disjointed and confusing that I finally had to scan the document and basically re- 
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write what you had done as there were too many corrections to make. I got 
about V 2 way through your 4 page document in that time. A lot of my corrections 
were guesses, because I really had a hard time trying to figure out the point you 
were trying to make. I asked you to write an essay, and this document looked 
like a summary in point form of one text book. 

This level of harshness was surprising and confronting, and the advisor (one of the 
authors of this paper) has since made efforts to rein in the frustration experienced with 
students who did not seem to be putting in the level of effort required. 

The third sub-theme that contributed to the process of postgraduate supervision was 
'engineering success' for the student. During an early iteration of the analysis, this block 
of activities was coded as "anxiety reduction" but this label did not reflect the extent to 
which the research process was engineered so that confidence emerged from within the 
students' own accomplishments rather than by simply experiencing a reduction in 
anxiety. Activities which contributed to 'engineering success' included encouragement, 
reframing, scaffolding and avoiding pitfalls. 

A related supervisory action focused on teaching the student to develop ways of 
conceptualising issues as internal sources of encouragement. In the psychology 
literature this technique is known as "reframing" and its purpose is to alter the meaning 
of something by altering its description. For example, an anxious student worried about 
an oral defence (confirmation) of her proposal was emailed: The main thing to 
remember is that there are only 4 hoops to jump through. Thus when your confirmation 
is completed you will have only 3: ethics approval from [The Institution], ethics approval 
from [the agency] and then your thesis examination. This means you are just about 1/4 
through your thesis. In this way the oral presentation became reframed as a hoop to 
jump through instead of a public judgement of a student's work. 

Another example of reframing is, 

I've skimmed your proposal and it is definitely on the right track. However, there 
is a kind of disorganisation of your ideas, so I've suggested that you write me a 
series of essays. As you send one to me I'll tell you what the next essay will be. 
The essays will pretty much only require you to cut and paste from the material 
you've sent me. The benefit of this activity is that it will keep your argument 
dear and not contaminate one idea with another one. 


The Product 

The product of the supervision relationship was the completed thesis, examination-ready 
and is conceptualised in Figure 2. This product is the result of three main interventions: 
content, style and finishing touches. 
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Figure 2. Elements of The Product 



About one third of the feedback to students related to basic skills that we thought should 
have been achieved at high school or at the undergraduate level. Paragraphing was the 
most common problem encountered. This included identification of a topic sentence and 
use of transitions between sentences and paragraphs. Typical examples of feedback 
included, 

Make sure that each sentence makes one point only [and] that the point relates to 
the topic sentence (first sentence) of each paragraph. 


And 


A non-sequitor is something that appears out of sequence. Sometimes you have 
a sentence that is not in the right place in a paragraph. When you see non- 
sequitor you need to search and find the sentence that is out of order. Then you 
decide to: 

a. ) Move it to another place In the paragraph. 

b. ) Delete it. 

c. ) Re-word it and keep it where it is. 

In addition to these basic skills, the supervisor also gave feedback on the more 
sophisticated scholarly style issues. This included development of an argument, such as 
"Be clear about the point of your argument. I think it is that [the existing] definitions are 
[no good] and your research is going to address the issue?" and being explicit: 
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Your conclusion doesn't appear to follow the previous statement. Try to say what 
you want very directly. Was this finding expected or unexpected? Why? How does this 
finding add to the literature? How does the literature help you to contextualize your 
findings? 

Feedback regarding finishing touches occurred predominantly at the write-up stage and 
included ensuring that there were no loose ends: "This is 1980s dollars. Is that low 
[income]? Poverty line? Can you find out what the mean/median income was back 
then?" 

Finally, feedback was provided about what we have termed 'window dressing', which 
included the use of appropriate terminology, "put in the term "essence" to describe the 
central theme of your research" and style, 

The content is great, but it is pretty bland and boring. Chapter 4 is expected to 
be boring, but chapter 5 is used to make up for that. There should be a hint 
(implicit, not explicit) of excitement and wonder about your findings. This is done 
using several techniques. 

-use of words: choose carefully 

-use of comparison: use words like "unlike ..." or "as expected ..." etc. 
-shorten sentences occasionally to add emphasis. 


and, 


Summarise a bit. Don't bother listing what was not significant; just say what was 
and say that results of all variables are in the appendix. That way you can get rid 
of the sub-headings (which break up the flow of what you are saying). 


The Investment 


Figure 3: Elements of The investment 

The inv estment. 


/ 

Thinking like a researcher 


\ 

Teaching for life 


■ 

Research methods 

I 


Scholarly Style 


The third main theme that emerged (Figure 3) was that a large part of the feedback 
concerned what we have termed the 'investment'. This is basically feedback that 
prepared each student for life after the degree has been conferred. Until we had 
completed the thematic analysis, the supervisor had no idea that she had been providing 
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feedback according to the students' professional aspirations. However, for some 
students who were looking to work in the health or social science field, constructs such 
as developing a professional niche/expertise was the focus of feedback: We have a lot of 
specialties [in your profession] but if you can make a case for [your research topic] as a 
speciality, you will have invented a job for yourself. 

For some students the investment entailed preparation for their lives as researchers or 
academics. Specific areas of feedback given to students with research career aspirations 
included the future application of essential skills they were learning: 

Once you have the basics under your belt you will find that it takes a lot less 
drafts to get the right "tone". Once you have this elusive skill you can supervise 
other students, write research articles and grant applications. It just takes 
practice like any other skill. 

and deeper explanations of style issues, such as to a married, male student: 

There is a rule in writing theses and proposals that a person never starts a 
paragraph with an author's name. The reason for this is as follows in a parable: 

You and your wife are having an argument. She says you never do any 
housework and you say she spends too much of your hard earned money. 

Neither of you argues by saying "Mary Jones said that you didn't do housework" 
or "Phil told me that you spent too much money". This is because the topic of the 
argument is not what Mary or Phii think or say... it is about housework and 
money. Therefore, your wife says things like "you never take out the rubbish. 

The last time you took out the rubbish was in 1984... You reply "and you are not 
spending the money on cleaning products. Last month you bought $90 worth of 
makeup". 

As you can see, each person says what they think/believe and then backs it up 
with evidence. This is the same format your proposal should take. Instead of 
saying "Jones (1999) says this"..., say "Qualitative studies have investigated this 
and left some unanswered questions. Brown (2000) highlighted this by his study 
which found xxx and Bloggs (2001) identified that yyy. This research will seek to 
clarify these issues". 

Text (word) Searching 

We next carried out a text search using words that kept cropping up during the thematic 
analysis. These included terms such as "examiners", "paragraphing" and "argument", 
which were the most common codes identified in the thematic analysis. 

The NVivo string search yielded results that helped us to identify specific students with 
problems, and to quantify the problems. This information was less helpful than the 
thematic search because the results focused on student attributes rather than on the 
supervisor's teaching style, the focus of reflective practice. For example, we used the 
term "topic sentence" as a text search through all the documents. The search resulted in 
the identification of five students who had received written feedback about topic 
sentences. As can be seen in table 1, "Annie" and "Robin" were the people who needed 
the most reminding about the use of a topic sentence. 

Thus the text search provided information about the specific learning needs of the 
students which could also potentially provide evidence for formal evaluation of a 
student's progress. We were also able to import the findings into SPSS and create a 
graph that highlights the differing abilities of students for each issue. However, the 
supervisor was already very aware of the problems Annie and Robin were having with the 
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concept of topic sentences, and the NVivo text search didn't provide her with any 
new information. It was also limited in that we might have used other words or phrases 
to explain the concept, and we still had to determine the context in which the term had 
been used; which could have been to praise, correct or just mention in passing. In order 
to address this limitation we carried out a third analysis using a text-pattern search. 


Table 1. Results of a text search for the term, "topic sentence" 


Student 

My comments pertaining to the search item "topic 
sentence" 

"Annie" 

1 . 

I'm not sure if the topic sentence of this paragraph 
matches the rest of the paragraph. 


2. 

State your topic sentence directly: "women commit just 
one tenth of all homicides, but over one third of child 
homicides". 


3. 

Get a clear topic sentence for this paragraph....What point 
are you trying to make? 


4. 

Clearer topic sentence: is this paragraph about 
perpetrator gender differences or victim age differences? 


5. 

Need a topic sentence 

"Craig" 

1 . 

Some organizational schemes are imposed on paragraphs 
to achieve a certain emphasis. The most common is the 
general-to-specific scheme, in which the topic sentence 
generally comes first and then the following sentences 
become increasingly specific. 

"Robin" 

1 . 

This topic sentence is too general. Make it reflect the rest 
of the paragraph content. 


2. 

Topic sentences: what is the main point of this 
paragraph? 


3. 

This topic sentence is focused on the articles. Change the 
sentences to reflect the point/argument you are making 
rather than the literature itself. 

"Jane" 

1 . 

This paragraph needs a topic sentence about nursing 
socialization being particularly stressful. 


2. 

Be less generic and more specific in the topic sentence of 
this paragraph. 

"Jill" 

1 . 

You need a topic sentence in each paragraph so that each 
paragraph makes only one point. 


2. 

Make sure that each sentence makes one point only that 
the point relates to the topic sentence (first sentence) of 
each paragraph. 


Pattern Search 

A text search of the term "argument" yielded the information that the supervisor had 
given feedback about developing an argument 17 times to Craig, 10 times to Jane and 8 
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times to Robin. Figure 4 provides details for all students for whom developing an 
argument was problematic. 


Figure 4. Results of a text search for "Argument" 



A Boolean search of "argument" OR "the point" yielded more hits than the string search, 
but the results were less delimited. There were an additional three hits for Craig, with 
only one being argument-based: "put in an introductory paragraph which explains the 
point you are going to make". However, this search also picked up two uses of the 
phrase "the point" in ways more conceptually related to technical writing aspects: "If you 
can't back up your point with a reference, you can't make the point in your proposal". 
Similarly, Robin gained another three comments focused on the argument and one on 
writing technique. Yvonne got one hit that was not related to feedback regarding the 
point of the argument. 

This led to a total of 52 hits for the pattern "argument or "the point", although five of 
these were conceptually incorrect. This demonstrates an error rate of 9.6%. In this 
exercise we were able to filter these false hits very easily, so they did not pose a validity 
problem. Our main issue was that for the purpose of facilitation of supervisor reflectivity, 
both the text and pattern searches were focused more on student learning issues than 
the advisor's teaching strategies. 


Discussion 

We found two unexpected things in undertaking this QDAS reflective activity. The first is 
that using a thematic analysis is a useful and productive form of self-reflection, whether 
or not qualitative software is used. The second is that reflective analysis using QDAS 
required us to be a great deal more thorough and methodical than the advisor usually is 
in practice. This is because, as Maclaran and Catterall (2002) explain, the program 
"forced" us to think about each piece of data. This led to new insights, especially in 
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relation to how individual activities relate to each other to form broad scopes of 
practice, and how blunt the advisor can occasionally be in her comments. 

Specific areas where the thematic analysis has suggested new strategies include the 
advisor's being too ready to correct grammar and avoid pitfalls. When we look at the 
pages of repeated feedback given to some students on points of generic 'undergraduate' 
skills, the teacher agrees that she was too quick to provide solutions; and is focusing on 
developing strategies to help students develop the ability to self-correct. These new 
strategies have now been incorporated into the 'system'. 


Limitations of This Research 

This analysis is based on the writing of one person, who also happened to be one of the 
people who conducted the analysis of feedback to students in only one (Australian) 
postgraduate system. Only written (usually email) feedback was analysed. This is 
essentially a one-way process that excluded analysis of data from the other party, in this 
case the student. We are sure that transcripts/observations of meetings with students 
would reveal new themes through mutual interactions. 

In addition, we do not view one supervisor's practice as that of an "every-advisor" since 
feedback to students is always influenced by age, gender, discipline and socio-political 
context. For example, recent funding models for Australian universities (with a focus on 
quick completions), has made research advisors become much more pragmatic about 
getting students "through" than was the case ten years ago. Thus the feedback analysed 
for this paper demonstrates a snapshot of one university teacher's more recent working 
life. 

From an epistemological perspective, analysing one's own feedback as research data can 
be viewed as a limitation of this paper because of the possibility of exacerbation of biases 
and blind spots (Lacity, 1994). Flowever, there is precedence in the literature for a case- 
study self-report approach (see for example,Benaquisto, 2000; McKenna, 2007; Spillett 
& Moisiewicz, 2004; Zuber-Skerritt & Roche, 2004), and some explicit support (Naimon, 
2000) for the practice. Other authors would argue that analysis of one's own writings 
increases validity by reducing the researcher-object distance (Cohen, 2005). In the end, 
it is up to each reader to decide the extent to which a similar endeavour will be of use to 
them. 

In terms of the use of QDAS for reflective deconstruction, the main limitation became 
evident in the text pattern search. The Boolean search results required careful analysis 
because the phrase "the point", for example, was used in various ways and had to be 
manually filtered. Although this pattern search did uncover several instances where a 
string search for "the argument" missed, for the purpose of reflectivity of teaching 
practice, we found both the string and pattern searches not particularly informative since 
they tended to provide information that we had already uncovered from the thematic 
analysis. It is as Welsh (2002) concludes, too easy to extend the coding beyond any real 
benefit. 


Conclusions 

Use of QDAS in a thematic analysis does yield valid, useful insights beyond what we think 
could have been achieved without this external structure. It could be especially useful in 
providing concrete evidence of practice for use by academics who might want to conduct 
such an analysis and present it for annual reviews. It can also indicate areas of strength 
and areas for improvement in broad categories such as this advisor's 'the process' or 
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more specific areas. In addition, this methodology is very adaptable for use in 
classroom, practica and clinical settings: wherever there is a reasonable quantity of 
written feedback. For example when the same advisor was teaching research methods 
courses, she categorised her feedback to students on their assignment (a research 
proposal) and found that the majority of her comments pertained to students' 
development of their argument. As a result, she added more activities related to this 
topic in the course. 

The efficiency of coding and developing models created by NVivo makes the process 
significantly quicker than a manual thematic analysis. The easy importation of 
documents makes reflective coding a tidy, straightforward process. However, for people 
unfamiliar with QDAS, it would be a very time consuming exercise to learn the program 
for a one-off endeavour. 

From a practical perspective, this exercise has led to a change on the behaviour of the 
research team when undertaking research supervision. The most significant change has 
been the decision on behalf of the advisor to avoid providing repetitive correction. 
Instead, for example, one correction is made with the following comment: 

I've skimmed your chapter 5 and it is very good. It contains the right 
information. However, there are some style issues that now need to be 
addressed. I've indicated these in the first page, so you can see the types of 
things I am talking about. Please read my comments and suggestions before you 
click on "accept changes". I need you to really understand why I have made the 
suggestions/changes so that you can keep editing the rest of the chapter yourself. 

It is anticipated that by approaching this aspect of research supervision in this manner, 
the student is enabled to further refine their writing skills without overly burdening the 
dissertation advisor. 


Recommendations 

Qualitative researchers, familiar with a qualitative analysis program should find this a 
straightforward method to identify and categorise skills they use in postgraduate 
supervision. On the other hand, people unfamiliar with thematic analysis and/or its 
software will need to consider whether the benefits of the activity are worth the costs of 
hours involved in learning the skills required. It is possible that the relatively easy 
automated coding using word or pattern searches may provide sufficient value for people 
who are not interested in carrying out an in-depth thematic analysis. 

Secondly, the thematic analysis, compared to the text searches, provides evidence of 
reflexivity and a means of conceptualising one's practice. However, it does require a 
relatively large amount of data in order for patterns to emerge. The main advantage of 
using QDAS, facilitation of manual handling of large amounts of data, would not be 
realised with limited textual data. Text and pattern searching is quicker and easier, but 
was of limited value for teacher self reflection of practice. However, if feedback from 
multiple teachers were subjected to a string/pattern search, this would be very helpful in 
identifying areas of common need amongst postgraduate students within a department. 

QDAS primarily facilitated an editing research style, focussing on data categorisation and 
exploration of patterns, rather than a holistic/interpretive style. It also did not take the 
teacher's thoughts, feelings and non-verbal actions into account. As a consequence, the 
results are far more delimited for reflective practice than analysis of a reflective journal 
would be; although for the express purpose of reflection of written feedback to students, 
it did lead to changes in practice regarding what feedback was provided and how it was 


h tips:/Idol org/l 0.20429/ijsotl.2008.020208 


14 



IJ-SoTL, Vol. 2 [ 2008 ], No. 2 , Art. 8 


presented. For a more holistic view, we suggest the inclusion of an electronic 
reflective journal as an additional data source in the analysis. 
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